Course - 16825 : Learning for 3D Vision
Name - Parth Nilesh Shah
AndrewId - pnshah
| Grid | Rays |
|---|---|
| Sample Points |
|---|
| Spiral Render | Depth |
|---|---|
| Spiral Render |
|---|
Box Center - (0.25, 0.25, 0.00)
Box Side Length - (2.00, 1.50, 1.50)
Note : No View Dependence
| Epoch 10 | Epoch 50 | Epoch 100 | Epoch 150 | Epoch 240 |
|---|---|---|---|---|
| Lego | Materials |
|---|---|
Trade-Offs
Incorporating view dependence enables Nerf to produce more realistic renders, capturing intricate effects like specular lights and nuanced shading variations. However, a potential drawback emerges as this addition raises the risk of Nerf overfitting to training views, potentially leading the model to associate colors and effects too closely with specific viewpoints rather than generalizing across the object structure. Consequently, novel view renderings may suffer, lacking the fidelity expected due to the network's unfamiliarity with these new directions.
Algorithm :
Code Implementation :
def sphere_tracing(self, implicit_fn,
origins, # Nx3
directions, # Nx3
):
points = origins
for i in range(self.max_iters):
distance = implicit_fn(points)
points = points + directions * distance
mask = implicit_fn(points) < 1
return points, mask
| Visualization |
|---|
| Input | Prediction |
|---|---|
Best Results (alpha = 10, beta = 0.05)
Hyperparameter Tuning --
Changing alpha, (beta = 0.05) -
| alpha = 50 | ||
|---|---|---|
| alpha = 10 | ||
| alpha = 1 |
Changing beta, (alpha = 10) -
| beta = 0.01 | ||
|---|---|---|
| beta = 0.05 | ||
| beta = 0.5 |
Analysis -
How does high beta bias your learned SDF? What about low beta?
High Beta -> smoother and diffused surface
Low Beta -> sharper surface with jagged edges
Which beta value is better for training with volume rendering?
High Beta.
Reason - higher beta leads to a smoother and broader surface leading to a higher overlap with the true surface helping the optimization convergence.
Lower beta might cause overfitting risks and unstable gradients making training difficult.
Would you be more likely to learn an accurate surface with high beta or low beta? Why?
Low beta helps in capturing sharp edges and fine details, thus making it better for learning a very accurate surface.
8.2 Fewer Training Views
| VolSDF | NERF | |
|---|---|---|
| 20 views |